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Executive Summary 



Performance Monitoring Results AprenDes - Year 1 (2004) 



Introduction 

This document summarizes the results obtained by the AprenDes project in 2004, the 
project’s first year of implementation. It provides the principal findings on program 
performance from a basehne in May 2004 to the end of the school year (late October 
2004). Progress on a number of project objectives related to decentralized school- and 
regional- based educational decision-making, teacher professional development and 
student academic achievement in rural multigrade schools is examined. The findings in 
the document are based principally on a report of GRADE, the external evaluation 
contractor, entitled Informe final de Resultados en Pruebas de Rendimiento, proyecto 
AprenDes. The findings of GRADE have been reanalyzed where appropriate to provide a 
Ml understanding of the progress made in meeting project objectives and are 
complimented by the findings of AprenDes evaluation personnel, who monitored project 
performance indicators related to national education policies during the year. 

Major bindings 

AprenDes was highly successful in implementing all components of the project 
during the first year of implementation. On indicators of process and impact the 
project had consistently higher results throughout the year than the comparison group. In 
two areas related to incorporation of local assessments into the national education 
decentralization process and the estabhshment of student governments that are 
contributing to community development, the AprenDes project exceeded projections for 
the four- year life of project. 

AprenDes students had consistently higher achievement in Communication and 
Mathematics throughout the 2004 school year than similar students in comparison 
schools. AprenDes students maintained between 2 to 29 average point advantages at the 
end of the school year in all subjects depending on the grade level studied. Eirst grade 
students had average gains 3.4% higher than comparison groups. 

Participation in the AprenDes program had a significant effect on student 
achievement at both the third- and sixth grade -level. When possible intervening 
variables, such as age, sex, mother’s education level and socio-economic status, program, 
pretest score, type of school, teacher experience and teacher salary were controlled, 
participation in the AprenDes program had a significant effect on achievement in third 
grade mathematics and communication, and sixth grade communication. 

The AprenDes program was equally effective for males and females. No significant 
differences by gender were found and the achievement advantages for AprenDes were 
similar for boys and girls in all subjects. 
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Student age was a significant factor in low achievement, especially in sixth grade. 

Older students had relatively lower scores in sixth grade, suggesting the importance of 
special emphasis in the learning of over age students in higher grades. 

AprenDes was highly successful in establishing student governments with diverse 
memherships that were engaged in community actions. By the end of the 2004 school 
year, student governments had been estabhshed in aU of the AprenDes schools of the 
sample and more than three- fourths of these school governments had a diverse 
membership and were involved in community development activities. 

AprenDes had mixed results in the implementation of active learning strategies 
during the 2004 school year. The project had generally higher levels of implementation 
of active learning strategies during the year than comparison groups. However, there was 
a shght decrease in the degree of implementation on the indicators over time. Such 
results are generally attributable to a lack of teacher confidence owing to insufficient 
follow-up. 

AprenDes supported-activities informed national policy and led to policy 
refinements related to the decentralization process. AprenDes provided technical 
assistance to Regional Education Directorate, which led to the approval of the Regional 
Council of Education, and funding for curriculum diversification, teacher training, and 
citizen participation through school councils. In addition, AprenDes worked closely with 
the Consejo Nacional de Educacion (National Education Council), to inform the Ministry 
of the results of regional assessments. Seventy-one percent of the actions, rather than the 
projected 33% were incorporated into Ministry pohcy efforts. 

Regionally, AprenDes contributed to increased local initiatives and to increased 
civil participation in school operations. The project worked with all 10 provincial 
educational administrative U nits (U nidad deGestion Educativa Local-UGEL) through 
quarterly meetings to provide orientation, collaborate with technical specialists of these 
units on training activities, and support the participation of local authorities 
international observation tours. In addition, the project provided technical assistance to 
schools in the development of school operation plans. These activities resulted in a 
higher percentage of actions than expected among local authorities and schools. For 
example, 36% of AprenDes school councils rather than the predicted 20% were found 
to be developing actions related to institutional education plan. 



Principal Recommendations 

Continue to implement program components that have resulted in positive 
programmatic and policy outcomes. These inelude teehnical assistanee and knowledge 
building for loeal and national authorities, support to eivil soeiety partieipation in sehool 
operations, support for diverse, aetion- oriented student governments, and emphasis on 
basie eompeteneies in mathematies and eommunieation at aU grade levels. 
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Increase focus on learning needs of overage students. Ideally the components of 
modular learning and flexible promotion should help to e lim inate overage students in 
project schools. However, in the short mn, a special study or review of the characteristics 
of overage students in the external evaluation contractor database can contribute to 
developing strategies to better serving the needs of such students. In addition, the 
analysis of internal efficiency of project schools should be completed as soon as data are 
available to inform strategies for addressing the needs of overage students. 

Review training and support procedures for teacher professional development in 
active learning methodologies. As necessary, strengthen follow up to encourage 
teaching strategies that foster student expression of opinions in class, use of small group 
teaching contexts, and use of learning materials. 

Consult with GRADE to revise projections on indicators in light of 2004 results and 
to refine formats for projecting progress on academic achievement indicators. This 
revision should include determining levels of mastery so that percentages of students 
obtaining mastery in mathematics and communication can be reported as well as mean 
test scores, and recalculation of projections where higher than expected first year 
progress has been made. 
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Performance Monitoring Results AprenDes - Year 1 (2004) 



I. Introduction 

This document describes the progress made on the performance indicators for the 
AprenDes project during 2004, the first year of project implementation. The information 
comes primarily from the work of the external evaluation contractor, GRADE, a Pemvian 
NGO with an excellent reputation in educational research and evaluation. GRADE’S 
work had two purposes. Eirst, the evaluation results will inform the implementation of 
AprenDes interventions. The results complement those of other monitoring and 
evaluation efforts conducted by the project to fine-tune the development of its 
educational program for rural multigrade schools. Second, the baseline and final 
evaluations conducted by GRADE served to estabhsh basehne indicators for measuring 
project results. As it is envisioned that such monitoring of results will take place on a 
yearly basis, the 2004 work provides a basis for systematic monitoring of performance 
over time. Thus, this work contributes to the measurement of final project results 
(Basehne indicators and projections of change over time from the Project Monitoring 
Plan are found in Attachment A). The findings of GRADE were complimented by the 
efforts of AprenDes evaluation personnel, who monitored project performance indicators 
related to national education pohcies during the year. 

This document was produced by AprenDes evaluation staff, which extrapolated the 
results from the final report on GRADE’S work, which was completed at the end of 
January 2005. The current report includes final 2004 results on ah indicators with the 
exception of completion, which owing to a remedial recuperation period for students that 
takes place in January/Eebmary of the fohowing year, are not available until March, 
when final statistics are sent to the UGEEs (Unidad de Gestion Local). 

II. Background 

The objective of the AprenDes project is to improve the performance of Peruvian primary 
school students. AprenDes promotes effective decentrahzation pohcies that increase the 
participation of local stakeholders in decision-making about educational quahty. The 
project focuses on mral multigrade schools where — because of poverty and isolation — 
children are at a disadvantage and have had little academic success in school. Increased 
academic success is being accomphshed through the development of model interventions 
that strengthen the management capacity of local schools and promote participatory 
active learning methodologies in schools and communities. The products of this work are 
the implementation of decentrahzation pohcies and pedagogical changes in primary 
schools and classrooms of the San Martin region, changes that wih lead to greater 
academic success for individual girls and boys. 

The project is designed to address priority needs of education in Pern, particularly in 
relation to educational decentrahzation pohcies and participatory management practices 
in mral schools. The project is assisting the Consejo Nacional de Educacion to develop 



6 




recommendations on successful decentralization practices applicable to the Pemvian 
reality, especially poor rural schools. Policy recommendations are being conveyed to the 
Ministry of Education through multiple channels for incorporation into the national 
pohcy dialogue on decentrahzation. 

In collaboration with the Ministry of Education, regional education officers, and school- 
level staff and constituencies, the project is transforming traditional schools into “active 
school communities” that exemphfy the use of active learning methodologies in the 
classroom and the active participation of parents and communities in classroom learning 
activities, school hfe, outreach, and management. These methodologies are designed to 
increase girls’ and boys’ academic achievement and leadership, their capacity to be 
proactive in managing and resolving social conflict, and their abihty to identify means to 
create a democratic society. Active learning methodologies are particularly effective in 
increasing girls’ school participation — including enrollment, completion, academic 
achievement, and abihty to actively contribute to development. 

During its first year, the project focused on 70 schools in all provinces of the department, 
with the aim of having the greatest impact on the primary school system across the entire 
region. The selected schools serve as the basis for project work, thereby facihtating later 
expansion to 70 “sister” schools in a network and eventuaUy affecting ah multigrade 
schools in San Martin. The goal is to have a demonstrable effect on the development of 
decentrahzed local management of quahty basic education, which the government can 
build on as it initiates its nation-wide decentrahzation of the education system in 2005. 

III. Results 

A. SO Indicators of Performance 

Strategic Objective 6: Local Management of Quality Basic Education Programs 
Implemented in Selected Geographic Areas 

Performance Indicator #1 : 

• MOE pohcies related to decentrahzed local management implemented in 
selected regions. 

Indicator Definition/M easure: 

• Number of MOE pohcies found in the recently completed Ley General de 
Educacion (“Eey”) of the new education decentr ahz ation law implemented in 
San Martin. 

AprenDes has held seminars and provided technical assistance to the Regional Education 
Directorate over the course of 2004. These efforts have focused on the approval of the 
Regional Council of Education, as cahed for in the new education decentr ahz ation law, 
and in assisting the Council and the Regional Directorate in developing a proposal for 
action in education that was submitted as part of the funding the request. The proposal 
has been approved and is third in terms of funding priorities. The areas of action in the 
proposal relate to curriculum diversification, teacher training, and citizen participation 
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through the Consejo Educativo Institucional (Institutional Education Council). In 
addition, AprenDes is working closely with the Consejo Nacional de Educacion 
(National Education Council), which was created under the law to provide independent 
pohcy support to the Ministry. Table 1 shows the areas of the new education 
decentr aliz ation law in which AprenDes is working. 

As can be seen in the table, the support of AprenDes in providing regional seminars, 
technical assistance in the design of the Profile, and fact-finding tours to other countries 
in Eatin America (El Salvador and Nicaragua) in the process of decentrahzation has 
contributed to four major MOE pohcy areas for decentrahzed local management being 
implemented during 2004. In addition, the AprenDes has supported actions of the 
Decentra liz ation Committee of the National Education Council. These actions wiU be 
discussed under Intermediate Result 1: Performance Indicator #1. 
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Table 1: Ministry Policies Related to Decentralized Local Management 
Implemented in Selected Regions 

Targeted Areas of the Law Regional Action 

Universalization, quality and equity 

Universalization - Intercultural, decentralized focus 
assuring equal opportunities and satisfactory achievement 
for all students. 

Education Quality - Guidelines on the goals of Peruvian profile of the Regional Education Project approved and funded 

education, basic curriculum that can be refined to meet (o carry out actions in curriculum diversification and teacher 

local needs; set minimum investment per student; ongoing training 

training of teachers; education as a public service career; 

adequate school infrastructure; research and development 

of innovations; harmonious institutional and human 

relationships 

Equity - compensatory policies; education projects to 
overcome inequities; assignment of resources to at-risk 
areas; measures to retain those at greatest risk; inclusive 
education curriculum; scholarships; literacy programs as 
needed; incentives to encourage teachers to work with 
socially vulnerable; bilingual intercultural education 
offered throughout the system. 

Structure of the Educational System 

Elexible to adapt to country’s diverse needs; collaboration 
among State, civil society, business and the media to 
assure relevance of services; MOE responsible for 
national curriculum and regional/local institutions for 
adaptations. 

The Education Community 

Representatives including students, teachers, parents. Profile of the Regional Education Project approved and funded 

community members, directors, will be members of the carry out actions in citizen participation through the 

Institutional Education Council and will be responsible Institutional Education Council, 
for the Institutional Education Project 

System Administration 

Regional Education Participatory Council will develop Government of San Martin approves the creation of the 
and implement and monitor the Regional Education Regional Education Council through a consultative process. 

Project; Institutional Education Councils will have the 
same function at the local level 

National Education Council 

An autonomous body to participate in the formulation. Council formed and engaged in regional assessments that are 

implementation, and monitoring of the National providing input to the educational decentralization process. 

Education Plan and provide input to medium and long- 
term planning. 

Source: AprenDes Project Monitoring Records, November 2004 

In relation to the projections made in the performance monitoring plan, AprenDes is on 
target. Table 2 shows that the project is half way to the eight MOE policies related to 
decentrahzed local management implemented in San Martin. 
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Table 2: Performance Data Projections: Number of MOE Policies in the “Ley 



General de Educacion” Implemented in San Martin 



Year 


Planned 


Actual 


2004 


2 


4 


2005 


2 




2006 


2 




2007 


2 





Performance Indicator #2 

Primary school completion rate in San Martin intervention schools 

No data are available for this indicator as school records on promotion, repetition, and 
dropout will not be available until March 2005, owing to a remedial recuperation period 
for students that takes place in January/Febmary of the following year. Calculations of 
cohort completion for 2004 require the end of year data on promotion, repetition, dropout 
and completion rates. 



Table 3: Performance Data Projections: Primary School Completion rate in San 

Martin Intervention Schools 



Year 


Planned 


Actual 


2004 


2% 


NA 


2005 


3% 




2006 


2% 




2007 


2% 





B. Intermediate Result I Indicators of Performance 

Intermediate Result I: Policies implemented to promote decentralized management 
of quality basic education programs 

Performance Indicator #1 : 

• Number of recommendations for reforms to promote educational 

decentralization with the Ministry of Education generated by the Consejo 
Nacional de Educacion (National Education Council - CNE). 



The National Education Council of Pern (Consejo Nacional de Educacion) began its 
actions in 2003 in helping to create the National Education Project: A Shared Vision for 
Pemvian Education in 2015, through dialogue with key actors in six representative 
regions of the country. At the same time, the Council initiated a process of formulating 
pohcy proposals on topics that the group considered fundamental for educational reform: 
early infancy; professional development of teachers; educational finance; 
decentrahzation; and educational quahty and equity. AprenDes provides financial 
support and technical assistance to the Council. 
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As can be seen in Table 4, the Couneil has made recommendations to the Ministry 
through five prineipal aetions: the written opinion of the Couneil on the regulations for 
the Edueation Decentrali z ation Law; A Proposal for Reeiproeal Commitments in 
Edueation; Pronouneement on the 2005 edueation budget; Studies to Support 
Decentrahzation; and Regional Dialogues. A number of reeommendations have been 
ineluded in eaeh aetion and several have resulted in Ministry aetions in response. 



Table 4: National Council Recommendations Made to MOE 



Council Action 


Recommendations to MOE 


MOE Response 


Opinion on Regulations 


Greater detail 


Rewrite of document 


Proposal for Commitments 


National mobilization for quality 


Formation of support group 




in integral communication 


Creation of national reading council 
(PROMOLIBRO) 

Law to fund local education iniatives (Fondo 
Nacional para el Desarrollo de la Educacion 
Peruana) 

Emergency program emphasizing 
reading/writing 




Strategy for teacher professional 


Commission of experts designing law for public 




development- merit pay increases 


service career in teaching 
Development of law on Accreditation, 
Certification and Evaluation of teachers 




Annual Education Budget 


Proposal that budget is increase to 6% of GNP 




increases 






Decrease corruption 


Campaign against cormption 
Congressional anti-corruption decree 


Pronouncement on 2005 budget 
-Coordination of key individuals 
supporting budget increase in 
newspapers on August 24, 2004 




Minister and Vice Minister proclaimed the 
expectation of increased funding for education 


Decentralization Studies 


Four studies: Commitment of 
actors in 8 regions; implications 
of the new law; decentralized 
financing and the National 
Education Project; and 
mechanisms to transfer resources 


No response as studies in progress 


Regional dialogues 


Dialogues with local leaders in 13 
regions 


No response as action still underway 



Source: AprenDes Project Monitoring Records, November 2004 



In relation to the projeetions made in the performanee monitoring plan, AprenDes is on 
target. Table 5 shows that the Couneil has earried out five distinet broad aetions, whieh 
have produeed a number of reeommendations. These are slightly over half of the eight 
aetions projeeted for the life of projeet. If aetions having responses are eonsidered, then 
three aetions have had direet responses from the Ministry. 
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Table 5: Performance Data Projections: National Education Council 



Recommendations made to MOE 



Year 


Planned 


Actual 


2004 


3 actions 


5 actions 


2005 


2 actions 




2006 


2 actions 




2007 


1 action 





Performance Indicator #2 

• Assessments of successful local education decentralization efforts 
Incorporated Into M Inistry of Education planning and policy 



In addition to the actions in favor of decentr aliz ation with basis in local reality initiated 
by the Council, there have been two actions that have resulted from efforts in San Martin 
that have been incorporated by the Ministry. These are San Martin’s proposal for 
Institutional Education Councils to broaden their participation and for San Martin to be 
declared a pilot area. The region has proposed that Councils not to be formed solely from 
members of a local Educational Network {Red Educativa), given the lim ited possibihties 
of these networks of fulfilling that function in their current state of development. The 
Ministry response has been to issue a Manual for Institutional Education Councils, which 
states the Council can be broadened to include appropriate local community members. 

San Martin has initiated, through meetings with MOE representatives, a proposal to be 
declared a pilot area for the Education Decentr aliz ation effort. The Ministry and the 
Regional government are currently working on the procedures for such an agreement. 

The National Council of Education had three of five assessments of successful local 
education decentralization efforts incorporated to some degree in MOE planning and 
pohcy. This 60% is again higher than the first year projections shown in Table 6. If the 
two local assessments engaged in by AprenDes with the Regional authorities and 
incorporated in MOE planning are included, a total of five of seven assessments have had 
an impact on MOE decentrahzation plans. This percentage (71%) is reflected in the 
table. 

Table 6: Performance Data Projections: Assessments of Successful Local Education 



Decentralization Ef 


forts Incorporated into MOE 


planning and policy 


Year 


Planned 


Actual 


2004 


33% 


71% 


2005 


40% 




2006 


43% 




2007 
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Performance Indicator #3: 

• School councils participating in school operations 

Indicator Definition/Measure 

• Percentage of project schools with consejos educativos institucionales, CEIs 
(school councils) developing proyectos educativos institucionales, PEIs 
(school operative projects) 

It was assumed that, as time is needed to form councils and encourage regular meetings 
before actual participation in planning takes place, no councils would be active at the 
basehne collected in May of 2004. Thus, data were collected on the number of councils 
formed, councils that had developed work plans, and the frequency of meetings, as weU 
as those that were actually developing a school operative project (proyecto educativo 
institucional-PEI). As can be seen from Table 7, AprenDes had a larger percentage of 
schools with councils (consejos educativos institucionales-CEI), a larger percentage of 
councils that had work plans, and a larger percentage of councils that reported meeting 
and could discuss meeting agenda than comparisons schools, throughout the year. In 
addition, the difference between AprenDes and Comparison schools generally grew 
during the year. 

Table 7: Change in School Council Performance during 2004 - AprenDes and 



Comparison Schools 



Indicator 


Baseline 


End of Year 


AprenDes 

(n=25) 


Comparison 

(n=50) 


Difference 


AprenDes 


Comparison 


Difference 


Change 

In 

Difference 


School 

Council 

Formed 


56% 


46% 


10% 


76% 


62% 


14% 


2% 


Council 

with 

Workplan 


28% 


16% 


12% 


32% 


18% 


14% 


2% 


Council 

holds 

Meetings 


56% 


44% 


12% 


60% 


22% 


38% 


26% 


Council 

Develops 

PEI 


24% 


08% 


16% 


36% 


08% 


28% 


12% 



Source: Cuetoetal Informe final: Evaluacion de impacto del primer ano del proyecto AprenDes, GRADE, January 
2005, Lima. 



In terms of the development of the school operations plan, data had been collected 
through open-ended questions, rather than exphcitiy asking about the PEI. As this type 
of planning was a new concept, the PEI itself wasn’t mentioned in open-ended responses. 
Thus, in the GRADE final report, the number of school councils developing a PEI was 
recorded as zero. However, respondents in answering the open-ended questions 
described activities, many of which were part of the requirements of the PEI. Thus, the 
AprenDes evaluation team consulted an expert of PEI development who reviewed 
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response categories and identified those that were activities required under the PEL As 
there were multiple categories, that category which was identified most often by both the 
AprenDes schools and by the comparison group in the most recent meeting 
(“implementation of materials and improvement of areas”) was used for consistency in 
measuring progress. As can be seen in Table 7, 24% of AprenDes schools had councils 
working in these school development planning activities early in the year. By the end of 
the year, this percentage had increased to 36%, while no change was found in the 
comparisons group. Thus, while an increase of 12% in AprenDes schools was found over 
the course of the year, this was shghtly below the projected levels. 

Table 8: Performance Data Projections: School Councils Participating in School 



Operations 



Year 


Planned 


Actual 


2004 


20% 


12% 


2005 


40% 




2006 


50% 




2007 


60% 





C. Intermediate Result 1.1: Indicators of Performance 

Intermediate Result 1.1: Institutional capacities improved to support and manage 
local education services 

Performance Indicator: 

• Local authorities (mayors, provincial directors, local council members, UGEL 
[provincial education directorate] directors) understand and support 
decentrahzation process 

Indicator Definition/Measure: 

• Percentage of local authorities demonstrating knowledge of MOE regulations 
on educational decentr aliz ation and carrying out actions prescribed therein 



AprenDes worked with the Regional Directorate, and the Regional Government 
providing technical assistance in the development of Regional Education Project 
(Proyecto Educativo Regional) and gathering information from regional organizations. 

The project has worked with all 10 provincial educational administrative Units (Unidad 
de Gestion Educativa Local-UGEL) through quarterly meetings to provide orientation on 
project activities and coordinate activities. Technical speciahsts of these units have 
worked with the project in training and other implementation activities. A number of the 
local authorities have participated in project- sponsored international observation tours 
designed to build knowledge about active learning programs and create international 
networks to further dialogue. 

As AprenDes activities gained a wider scope over the year, more local authorities were 
involved. Thus, the sample of authorities interviewed expanded from 43 individuals at 
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the baseline to 56 at the end of the year. 



Table 9: Local Authorities with Knowledge of National Education Decentralization 
Policies and Supporting Policies with Actions 



Indicator 


Baseline (n=43) 


End of Year (n=56) 


Average percentage of 
authorities showing 
knowledge across all national 
educational policies 


71% 


75% 


Percent of authorities showing 
knowledge of national 
decentralization policies 


84% 


82% 


Average percentage of 
authorities engaged in actions 
to further all national 
educational policies 


37% 


38% 


Percent of authorities 
engaged in actions to further 
national decentralization 
policies 


26% 


21% 



Source: Cuetoetal Informe final: Evaluacion de impacto del primer ano del proyecto AprenDes, GRADE, January 
2005, Lima. 



Results for aU authorities shown in Table 9, were compared to those for only authorities 
who responded at the baseline and the end of the year. Similar tendencies were found 
with both groups. This suggests that the slight decreases in actions related to 
decentrahzation is not the result of new individuals being added to the sample but rather 
to completion of an activity that was underway early in the year. However, as such an 
assumption cannot be verified, the lower percentage was used for reporting purposes. 



Table 10: Performance Data Projections: Local Authorities Understand and 
Support Decentralization Process 



Year 


Planned 


Actual 


2004 


20% 


21% 


2005 


40% 




2006 


45% 




2007 


50% 





D. Intermediate Result 2: Indicator of Performance 



Intermediate Result 2: Local basic education programs in selected communities 
improved. 

Performance Indicator: 

• Student mastery of subject matter 

Indicator Definition/Measure: 
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• Percentage change in mean reading scores 

• Percentage change in students demonstrating mastery of the curriculum 



Tests in communication s kill s (comunicacion integral) were administered to a 
representative sample of male and female students in first, third and sixth grade of 
AprenDes schools and comparison schools in May and November of 2004. In addition, 
tests in mathematics achievement were administered to the same students in third and 
sixth grade. Tests were administered at the start of the school year in order to monitor 
gains and to estabhsh a basehne on which to monitor progress in subsequent years. 

1 . First Grade 

Table 11 presents the overall results of the pre-reading test administered to students in 
AprenDes schools, “sister” schools, which will enter the AprenDes program in 2005, and 
other comparison school not yet scheduled to enter the program. Table 1 1 is the 
summary of 24 comparisons on individual items. In those comparisons, AprenDes 
students had higher percentage of correct responses on 20 of the 24 items. Similarly, 
when the increment of change was examined, AprenDes first were higher on 15 of the 24 
comparisons. This is reflected in the table in that AprenDes has higher total average 
post- test scores on each of the five dimensions and has a higher increment of change on 
four of the five overall comparisons. When all dimensions are totaled, AprenDes has a 
higher mean average percentage correct across all dimensions (29.9 versus 24.7) and a 
higher mean average percentage of change from the beginning of the school year. While 
the percentages are not statistically significant the consistency of the results demonstrate 
the positive impact of AprenDes. 



Table 11: First Grade Student Achievement in Reading: Average Percent Correct 

on Each Dimension and Total Correct 



Periods 


May 


November 


Change 


Dimensions 


AprenDes 


Comparison 


AprenDes 


Comparison 


AprenDes 


Comparison 


ID 

V owels/Consonants 


57.8-b 


48.2-1- 


67.5-b 


58.5-b 


9.8 


10.3 


Initial Writing 


17.9 


16.8 


59.1-b 


48.8-b 


41.2-1- 


32.0-b 


Initial Reading 


11.9 


10.0 


40.9 


33.8 


29.0 


23.7 


Written Concepts 


27.7 


22.6 


40.6 


33.5 


12.9 


10.9 


Total Pre-reading 


12.5 


10.6 


29.9 


24.7 


17.5 


14.1 



Note: “Comparison” includes both Sister and Control schools; +significant at p < .1; *significant at p < .05; 
**significant at p < .01 

Source: Cueto et al. Informe final de Resultados en Pruebas de Rendimiento, proyecto AprenDes, GRADE, December 
2004, Lima. 



AprenDes also had a higher percentage of students who qualified to take the reading 
comprehension test for first grade (28% versus 24% for Sister schools and 19% for 
comparison schools). On the actual tests no significant differences were found, as the 
screen process lim ited the test sample to children with similar ski ll s. 
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Given the importance of first grade reading to the Regional authorities and to the 
Ministry of Education’s plan de emergencia, (emergency plan), first grade progress was 
chosen for the reporting indicator in year 1. As can be seen from Table 12, AprenDes 
surpassed its projections for 2004 on this indicator. 

Table 12: Performance Data Projections: Percentage Change in Mean Reading 



Score 



Year 


Planned 


Actual 


2004 


1% 


3.4% 


2005 


1% 




2006 


3% 




2007 


5% 





2. Third Grade 

As shown in Tables 13 and 14, AprenDes third grade students demonstrated higher 
achievement in both Mathematics and Communication. AprenDes students had an 
average end-of year scores that ranged between 18 and 29 points higher than those of 
their peers in non- AprenDes schools on all comparisons in Mathematics. Similarly, 
AprenDes students showed average mean differences of between 10 and 29 points in 
Communication. 

AprenDes students also showed significant learning gains in both subjects during the 
year. However, these gains were not significantly different from those found in 
comparison schools. Thus, multivariate analysis was used to control for possible 
intervening variables. These variables included both individual student factors, such as 
age, sex, mother’s education level and socio-economic status, as well as school and 
teacher level factors, such as program (AprenDes or comparison), pretest score, type of 
school (one teacher, two teachers, three teachers), teacher experience and teacher salary. 
As shown in the tables, the effect of program, when a school was AprenDes, had a 
significant effect on student achievement in both subjects (Documentation of the 
multivariate analyses is found in Attachment C). No differences by gender were found 
either within or across programs. 



Table 13: Third Grade Student Achievement in Mathematics 



Program 




End of Year Score 


AprenDes Program Effect 


AprenDes 


321.1* 


39.5** 


Sister Schools 


291.3 




Control Group 


302.4 





+significant at p < .1; *significant at p < .05; **significant at p < .01 

Source: Cueto et al. Informe final: Evaluacidn de impacto del primer ano del proyecto AprenDes , GRADE, January 
2005, Lima. 
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Table 14: Third Grade Student Achievement in Communication 



Program 




End of Year Score 


AprenDes Program Effect 


AprenDes 


327.2* 


34.8** 


Sister Schools 


297.9 




Control Group 


317.3 





+significant at p < .1; *significant at p < .05; **significant at p < .01 

Source: Cueto et al. Informe final: Evaluacion de impacto del primer ano del proyecto AprenDes , GRADE, January 
2005, Lima. 



Sixth Grade 

As can be seen in Tables 15 and 16, the similar tendencies were found for sixth grade 
performance in terms of mathematics and communication achievement. AprenDes 
students had higher achievement early in the year and this advantage was maintained 
over the course of the year. Differences favoring AprenDes were almost 13 points in 
mathematics and over 20 points in Communication. However, in mathematics the 
difference with the comparison groups was not significant. In communication, 
significant differences in achievement were maintained through the year. There was no 
difference in results by gender and program effects were found to be significant in 
communication and tending toward significance in mathematics. 



Table 15: Sixth Grade Student Achievement in Mathematics 



Program 




End of Year Score 


AprenDes Program Effect 


AprenDes 


513.3 


17.9+ 


Sister Schools 


500.4 




Control Group 


500.4 





+significant at p < .1; *significant at p < .05; **significant at p < .01 

Source: Cueto et al. Informe final: Evaluacion de impacto del primer ano del proyecto AprenDes , GRADE, January 
2005, Lima. 



Table 16: Sixth Grade Student Achievement in Communication 



Program 




End of Year Score 


AprenDes Program Effect 


AprenDes 


523.9* 


22.9* 


Sister Schools 


503.0 




Control Group 


502.0 





+significant at p < .01; *significant at p < .05; **significant at p < .01 

Source: Cueto et al. Informe final: Evaluacion de impacto del primer ano del proyecto AprenDes , GRADE, January 
2005, Lima. 



AprenDes students had significantly higher achievement in Communication and 
Mathematics throughout the 2004 school year than similar students in comparison 
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schools. This achievement was related to participation in the AprenDes project when 
other intervening variables were controlled. When other variables were controlled, there 
was a general effect for age, with older children in sixth grade tending to have lower 
scores across schools 

E. Intermediate Result 2.1: Indicator of Performance 

Intermediate Result 2.1: Effective teaching and appropriate materials combined to 
support learning 

Performance Indicator 

• Teachers use active learning methods appropriately 
Indicator Definition/Measure 

Percentage change in a teaching quality index consisting of 1) number of boys 
and girls expressing their opinion in class; 2) use of small group methodology; 3) 
use of education materials by students; 4) number of parents reviewing student 
performance) aggregated across sample classrooms, expressed as a value between 
0 (rninimum) and 1 (maximum). 

In the original AprenDes performance monitoring plan, it had been determined that use of 
libraries would be included in the teaching quahty index because of the hbrary’s 
importance in the AprenDes instmctional process. However, information on the hbrary 
was collected by asking school directors if children used the hbrary, rather than asking 
the percentage of students that used the library. Therefore, no common denominator 
exists to include these data with other index measures. Thus, use of materials by 
students, obtained by GRADE through stmctured observations has been substituted for 
the original measure. 

Table 17: Change in Teachers Use of Active Learning Methodologies - AprenDes 



and Comparison Schools 



Indicator 


Baseline 


End of Year 


AprenDes 

(n=25) 


Comparison 

(n=50) 


Difference 


AprenDes 


Comparison 


Difference 


Change 

In 

Difference 


Percent of 
Student 
Opinions 
in 

Teacher- 

Student 

Interaction 


35% 


34% 


1% 


32% 


41% 


-9% 


-8% 


Percent of 
Small 
Group 
Activity 


41% 


19% 


20% 


45% 


24% 


21% 


1% 


Percent of 
Students 


72% 


48% 


24% 


68% 


57% 


11% 


-13% 
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Using 

Materials 
















Percent of 

parents 

Reviewing 

Student 

Progress 


47% 


41% 


6% 


47% 


45% 


1% 


-5% 


Index of 

Active 

Learning 


49% 


35% 


13% 


49% 


42% 


7% 


-6% 



Source: Cuetoetal Informe final: Evaluacion de impacto del primer ano del proyecto AprenDes, GRADE, January 
2005, Lima. 



As can be seen from Table 17, AprenDes elassrooms began the sehool year with 
favorable differences on all of the measures of active learning. However, although this 
favorable difference was maintained on three of the four indicators, percentages 
decreased on two of the four and remained eonstant on the fourth indicator. In the 
comparison groups, on the other hand, there were gains on all indicators. Such results are 
often common in the first year of educational innovation in the classroom. Generally a 
decrease in the use of innovative teaching strategies is caused by teacher lack of 
confidence owing to insufficient follow-up. Teachers begin the year with confidence in 
the new approaches but if may regress to traditional methods, if not given sufficient 
reinforcement. This would seem to be the case with AprenDes, as teaehers, in general 
continued to use the methods more than the comparison groups but decreased in their 
overall usage. 



Table 18: Performance Data Projections: Teachers Use Active Learning Methods 



Appro] 


priately - AprenDes and Com] 


parison 


Year 


Planned 


Actual 


2004 


10% 


-6% 


2005 


10% 




2006 


10% 




2007 


60% 





E. Intermediate Result 2.2: Indicators of Performance 

Intermediate Result 2.2: Democratic behaviors and valuing diversity practiced by 

students, parents, and teachers 

Performance Indicator #1 

• Student governments {Municipios escolares) formed and earrying out student 
initiatives in projeet sehools 

Indicator Definition/Measure 

• Pereentage of all projeet sehools with student governments that have a 
diversity of members in terms of age, sex, ethnieity, and length of residenee in 
the area, and whieh have organized initiatives 
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• Percentage of all project schools with student governments that are carrying 
out projects related to community development and healthy hfe styles 



As shown in Table 19, AprenDes was able to establish student governments in a majority 
of its schools even early in the school year. This is different than the comparison schools, 
where through the year, student governments were only estabhshed in 10% of the 
schools. By the end of the year, student governments were established in aU of the 
AprenDes schools in the sample. Seventy- nine percent of these student governments had 
female members, members from lower grades, and members from different provinces 
than that in which the school was located. The percentage of schools carrying out 
community development activities also increased as more AprenDes student governments 
were estabhshed. Early in the year only 20% of student governments were engaged in 
community development activities, whereas by the end of the year the figure was 80%. 

In the comparison group, on the other hand, almost ah of the schools with student 
governments carried out such activities but governments only existed in five schools. 

The activities undertaken by the student governments were generaUy awareness 
campaigns about the environment and health, community beautifying efforts, and care of 
community natural resources. 

Table 19: Change in Schools with Student Governments - AprenDes and 



Comparison Schools 



Indicator 


Baseline 


End of Year 


AprenDes 

(n=25) 


Comparison 

(n=50) 


Difference 


AprenDes 


Comparison 


Difference 

AprenDes 


Difference 

Comparison 


Percent of all 
Schools with 
Student 
Governments 


64% 


10% 


54% 


100% 


10% 


36% 


0% 


Percent of all 
Schools with 
Diverse 
Representation 
in 

Governments 


44% 


6% 


38% 


79% 


8% 


35% 


2% 


Percent of all 
Schools with 
Governments 
Carrying out 
Community 
Projects 


20% 


8% 


12% 


80% 


8% 


60% 


0% 



Source: Cueto et al Informe final: Evaluacidn de impacto del primer ano del proyecto AprenDes, GRADE, January 
2005, Lima. 



It was assumed that some time would be needed to organize democratic selection of 
student governments that would ensure diverse representation. Thus, projections for the 
first year were set at 20% of the total number of AprenDes schools. However, there was 
great interest in this component of the program among teachers and students and it 
developed more rapidly than expected. As can be seen from Table 20, AprenDes reached 
the projection for the four- year hfe of the project in the first year. 
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Table 20: Performance Data Projections: AprenDes Schools with Diverse Student 

Governments 



Year 


Planned 


Actual 


2004 


20% 


79% 


2005 


35% 




2006 


50% 




2007 


70% 





As with diverse representation, community involvement among AprenDes student 
governments proceeded more rapidly than projected. Again, as can be seen from Table 
21, the life of project projections were surpassed in the first year of implementation. 



Table 21: Performance Data Projections: AprenDes Schools with Student 



Governments Engaged in Community Actions 



Year 


Planned 


Actual 


2004 


10% 


80% 


2005 


20% 




2006 


30% 




2007 


50% 





22 
































rv. Conclusions and Recommendations 



A. Conclusions 

1. General 



AprenDes was highly successful in implementing all components of the project 
during the first year of implementation. On indicators of process and impact the 
project had consistently higher results throughout the year than the comparison group. In 
two areas related to incorporation of local assessments into the national education 
decentrahzation process and the estabhshment of diverse student governments that 
contribute to community development, the AprenDes exceeded projections for the four- 
year hfe of project. 

2. Student Achievement 

AprenDes students had consistently higher achievement in Communication and 
Mathematics throughout the 2004 school year than similar students in comparison 
schools. AprenDes students maintained between 2 to 29 average point advantages at the 
end of the school year in all subjects depending on the grade level studied. First grade 
students had average gains 3.4% higher than comparison groups. 

Participation in the AprenDes program had a significant effect on student 
achievement at both the third- and sixth grade -level. When possible intervening 
variables, such as age, sex, mother’s education level and socio-economic status, program, 
pretest score, type of school, teacher experience and teacher salary were controlled, 
participation in the AprenDes program had a significant effect on achievement in third 
grade mathematics and communication, and sixth grade communication, when compared 
individually to “sister” schools and comparison schools. 

The AprenDes program was equally efl’ective for males and females. No significant 
differences by gender were found and the achievement advantages for AprenDes were 
similar for boys and girls in all subjects. 

Student age was a significant factor in low achievement, especially in sixth grade. 

Older students had relatively lower scores in sixth grade, suggesting the importance of 
special emphasis in the learning of over age students in higher grades. 

3. Policy Support 

AprenDes supported-activities informed national policy and led to policy 
refinements related to the decentralization process. AprenDes held seminars and 
provided technical assistance to Regional Education Directorate, which led to the 
approval of the Regional Council of Education, and to the Council obtaining funding for 
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curriculum diversification, teacher training, and citizen participation. In addition, 

AprenDes worked closely with the Consejo Nacional de Educacion (National Education 
Council), to inform the Ministry of the results of regional assessments. Seventy-one 
percent of the actions, rather than the projected 33% were incorporated into Ministry 
pohcy efforts. 

Regionally AprenDes contributed to increased local initiatives and to increased civil 
participation in school operations. The project worked with all 10 provincial 
educational administrative Units (JJnidad de Gestion Educativa Local-UGEL) through 
quarterly meetings to provide orientation on project activities and coordinate activities, 
technical speciahsts of these units participated in AprenDes training, and a number of the 
local authorities participated in project- sponsored international observation tours. In 
addition, the project provided technical assistance to schools in the development of 
school operation plans. These activities resulted in a higher percentage of actions than 
expected among local authorities and schools. 

4. Active Learning Methodologies 

AprenDes was highly successful in establishing student governments with diverse 
memberships that were engaged in community actions. By the end of the 2004 school 
year, student governments had been estabhshed in aU of the AprenDes schools in the 
sample. An average of 79% of these schools had members that were female, that were in 
the lower grades, and that were not from the province in which the school was located. 

Eighty percent of the school governments were involved in community development 
activities. 

AprenDes had mixed results in the implementation of active learning strategies 
during the 2004 school year. The project maintained a favorable difference on three of 
the four indicators of active learning throughout the school year. However, there was a 
decrease in the degree of implementation on two of the four and the fourth remained 
constant. In the comparison groups, on the other hand, there were gains on aU indicators. 

Such results are generally attributable to a lack of teacher confidence owing to 
insufficient follow-up. 

5. Monitoring of AprenDes Progress 

Results on 2004 indicators showed that adjustments in measures and projections are 
needed in some cases. In student achievement, while the results presented positive 
trends for the AprenDes project, the complexity of the data analysis meant that the results 
do not lend themselves to the reporting formats originally proposed in the Project 
Monitoring Plan. On several indicators, such as civil participation in school operations, 
active learning methodologies, and pohcy support, assumptions used to make original 
projections were found to be inaccurate. 

Change in internal efficiency is an important indicator and first year findings are 
important to the refinement of project implementation strategies. Given the 
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difficulties found for overage students, it is important to determine student progress 
through primary school in order to develop interventions to improve efficiency, where 
necessary. 

B. Recommendations 

Continue to implement program components that have resulted in positive 
programmatic and policy outcomes. These include technical assistance and knowledge 
building for local and national authorities, support to civil society participation in school 
operations, support for diverse, action- oriented student governments, and emphasis on 
basic competencies in mathematics and communication at all grade levels. 

Increase focus on learning needs of overage students. Ideally the components of 
modular learning and flexible promotion should help to e lim inate overage students in 
project schools. However, in the short mn, a special study or review of the characteristics 
of overage students in the external evaluation contractor database can contribute to 
developing strategies to better serving the needs of such students. 

Review training and support procedures for teacher professional development in 
active learning methodologies. As necessary, strengthen follow up to encourage 
teaching strategies that foster student expression of opinions in class, use of small group 
teaching contexts, and use of learning materials. 

Through consultation with GRADE, revise projections on indicators in light of 2004 
results and refine formats for projecting progress on academic achievement 
indicators. This revision should include determining levels of mastery so that 
percentages of students obtaining mastery in mathematics and communication can be 
reported as well as mean test scores, and recalculation of projections where higher than 
expected first year progress has been made. 

Complete the analysis of change in internal efficiency as soon as data are available. 

The difficulties found for overage students, suggest that it is important to track changes in 
student progress through primary school. Thus, a basehne should be estabhshed as soon 
as possible. 
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ATTACHMENT A: SUMMARY OF 2004 INDICATORS 
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AID INDICATORS - APRENDES PROJECT 




Baseline 

Year 

Data 



FY FY 

2005 20006 

Actual Target 



% of school 

councils in 

USAID-financed 

schools 

conducting 

school 

planning 



% change in 
mean scores of 
standardized tests 



% change m a 
teaching quality 
index for active 
learning 
methodologies 



Number of MOE 
policies found in 
the recently 
completed Ley 
General de 
Educacion (Ley 
General) of the 
new education 
decentralization 
law implemented 
in San Martin 



% of children 
who complete 
primary school in 
the normal 
primary cycle 



Number of 

recommendations 

for reforms to 

promote 

educational 

decentralization 

with the Ministry 

of Education 

generated by the 

Consejo 

Nacional de 

Educacion 

(National 

Education 

Council - CNE) 



Percentage of 
CNE- 

recommended 
reforms that are 
incorporated into 
ministry 
pianning and 
poiicy documents 



Percentage of 
local authorities 
demonstrating 
knowledge of 
MOE regulations 
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on educational 

decentralization 

and carrying out 

actions 

prescribed 

therein 


























Percentage of all 
project schools 
with student 
governments that 
have a diversity 
of members in 
terms of age, sex, 
ethnicity, and 
length of 
residence in the 
area, and which 
have organized 
initiatives 




44% 










20% 


79% 


35% 








Percentage of all 
project schools 
with student 
governments that 
are carrying out 
projects related 
to community 
development and 
healthy life styles 




20% 










10% 


80% 


60% 









*Target in AprenDes USAID approved PMP (July 2004) is 1% 

**Target for FY2004 was first grade given Ministry focus on reading in this grade. 
However in multivariate analyses there were also significant program effects on 



student achievement in communications in third grade and sixth in AprenDes schools 
***Target in AprenDes USAID approved PMP is 10% 
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